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DETAILED ACTION 

1. Claims 1-2, 4-14, 16-23, 25 are pending in this application. Claims 1, 14, 21 are 
independent claims. This action is made Final. 



Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the invention was 
made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 

This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1 .56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 

3. Claims 1, 2, 4-8, 10-14, 16-23, 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yamaguchi (US Patent No. 7,006,671), in view of Toyama et al. (US Patent 
No. 7,068,309). 

As to claims 1,14, Yamaguchi teaches a method of capturing, analyzing, managing, and 
accessing disparate types and sources of media, biometric, and database information, the method 
comprising (col. 2, line 20 to col. 3, line 33; Figs. 2, 3A, 3B); 

capturing media, biometric, and database information association with an individual (i.e. 
information related to a bodily feature such as a fingerprint, voiceprint, hair, or nail can also be 
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used as specific information, or a physical key such as an IC card can also be used as specific 
information. Security can be improved using a plurality of pieces of specific information, col. 3, 
line 63 to col. 4, line 4); 

including time information with the captured media, biometric, and database information 
associated with an individual to create a multi-modal chronological dossier of the individual, 
wherein the time information includes when the media, biometric, and database information is 
captured (i.e. for each registered person, pieces of information including the captured image or 
extracted feature as the specific information of the registered person, the dictionary, the name of 
the registered person, the registration number, the registration date when these pieces of 
information are stored in the storage device 11, and the use situation such as the date/time and 
place of identification of the registered person, col. 5, lines 31-54, col. 2, line 20 to col. 3, line 
33; Figs. 2, 3 A, 3B; and col. 5, line 61 to col. 6, line 3); 

processing the media, biometric, and database information to extract, analyze and sort 
through digital information associated with a number of individuals (i.e. extract specific 
information of an object person from the object person , col. 2, lines 3-45; the sorting method, 
col. 5, line 61 to col. 6, line 3); 

providing a user interface (i.e. The administrator can easily obtain necessary information 
at a necessary time, col. 5, lines 31-54) that can be configured to retrieve, view, manage, 
compare, and analysis (i.e. classifies; comparing, sorts, col. 2, lines 3-45). 

Yamaguchi does not specifically teach annotate the captured information. 

However, Toyama teaches annotate the captured information (col. 3, lines 38-50; col. 4, 
lines 49-67). 
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It would have been obvious to one of ordinary skill of the art having the teaching of 
Yamaguchi and Toyama at the time the invention was made to modify the system of Yamaguchi 
to include annotate the captured information as taught by Toyama. 

One of ordinary skill in the art would be motivated to make this combination in order to 
allow the user to effectively search and control access to the photo index or photo database in 
view of Toyama, as doing so would give the added benefit of easily sharing images with a 
specific small group of people, a larger group of people, or the whole world as taught by Toyama 
(col. 3, lines 25-37). 

As per claim 21, Yamaguchi teaches a processing system comprising: 
a central processing unit (CPU) (Fig. 1, col. 10, lines 16-19); and 
a storage device coupled to the CPU and having stored there information for configuring 
the CPU to: 

capture media, biometric, and database information associated with an individual 
(i.e. information related to a bodily feature such as a fingerprint, voiceprint, hair, or nail can 
also be used as specific information, or a physical key such as an IC card can also be used as 
specific information. Security can be improved using a plurality of pieces of specific information, 
col. 3, line 63 to col. 4, line 4); 

assign timing information (i.e. timepiece 18, Fig. 1) to the captured media, 
biometric, and database information associated with the individual to form a history of captured 
information including times of when the media, biometric and database information is captured 
(i.e. for each registered person, pieces of information including the captured image or extracted 
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feature as the specific information of the registered person, the dictionary, the name of the 
registered person, the registration number, the registration date when these pieces of 
information are stored in the storage device 11, and the use situation such as the date/time and 
place of identification of the registered person; "images of persons who used the personal 
identification apparatus from X to Y o'clock", col. 5, lines 31-54); 

processing the media, biometric, and database information to extract, analyze and 
sort through digital information associated with a number of individuals (i.e. extract specific 
information of an object person from the object person , col. 2, lines 3-45; the sorting method, 
col. 5, line 61 to col. 6, line 3); 

providing a user interface (i.e. The administrator can easily obtain necessary 
information at a necessary time, col. 5, lines 31-54) that can be configured to retrieve, view, 
manage, compare, and analysis (i.e. classifies; comparing, sorts, col. 2, lines 3-45). 
Yamaguchi does not specifically teach annotate the captured information. 
However, Toyama teaches annotate the captured information (col. 3, lines 38-50; col. 4, 
lines 49-67). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Yamaguchi and Toyama at the time the invention was made to modify the system of Yamaguchi 
to include annotate the captured information as taught by Toyama. 

One of ordinary skill in the art would be motivated to make this combination in order to 
allow the user to effectively search and control access to the photo index or photo database in 
view of Toyama, as doing so would give the added benefit of easily sharing images with a 
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specific small group of people, a larger group of people, or the whole world as taught by Toyama 
(col. 3, lines 25-37). 

As per claim 2, Yamaguchi teaches the media, biometric, and database information 
includes a facial image, voice audio, or fingerprint (col. 3, line 63 to col. 4, line 4). 

As to claims 4, 16, Yamaguchi teaches forming a summary profile that is an abstract 
including intelligent portion of various captures of media, biometric, and database information 
associated with the individual (Figs. 2, 3A-3B). 

As to claims 5, 17, Yamaguchi teaches selectively presenting the summary profile on the 
user interface (col. 9, lines 62-64). 

As to claims 6, 18, Yamaguchi teaches the selective presentation of the summary profile 
in the user interface is in response to a search query (col. 5, lines 31-54; col. 9, lines 45-53). 

As to claims 7, 19, Yamaguchi teaches providing for a user-defined search of digital 
information associated with a number of individuals (col. 5, lines 31-54). 

As to claims 8, 20, Yamaguchi teaches conducting a more like this search when a search 
result from the user-defined search of digital information associated with a number of individuals 
is explored (col. 5, lines 31-54). 
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As per claim 10, Yamaguchi teaches capturing media, biometric, and database 
information associated with an individual includes using a video camera to capture audio and 
moving pictures of the individual (col. 4, lines 5-67). 

As per claim 11, Yamaguchi teaches processing the media, biometric, and database 
information to extract, analyze and sort through digital information associated with a number of 
individuals includes analyzing the media, biometric, and database information with respect to 
identification factors (col. 2, line 20 to col. 3, line 33; Figs. 2, 3A, 3B). 

As per claim 12, Yamaguchi teaches processing the media, biometric, and database 
information to extract, analyze and sort through digital information associated with a number of 
individuals includes comparing captured media, biometric, and database information of a first 
individual with media, biometric, and database information of a number of categorized 
individuals to find a best match (col. 2, line 20 to col. 3, line 33; Figs. 2, 3A, 3B). 

As per claim 13, Yamaguchi teaches displaying video thumbnails of video images of the 
number of individuals on the user interface (Fig. 6). 

As per claim 22, Yamaguchi teaches a presentation device wherein the presentation 
device is configured to provide a graphical user interface which presents representations of the 
captured media, biometric, and database information associated with the individual {col. 5, lines 
31-54). 
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As per claim 23, Toyama teaches an interface device configured to connect the CPU 
with a network of computers (col. 3, lines 25-37). 

As per claim 25, Yamaguchi teaches the CPU is further configured to form a summary 
profile that is an abstract including intelligent portions of various captures of media, biometric, 
and database information associated with the individual (Figs. 2, 3A-3B). 

4. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yamaguchi (US 
Patent No. 7,006,671), in view of Toyama et al. (US Patent No. 7,068,309), and further in view 
of Wilson et al. (US Patent No. 6,047,282). 

As per claim 9, Yamaguchi teaches the more like this search uses facial, and other 
biometric information to find matches (col. 5, lines 31-54). 

Yamaguchi and Toyama do not explicitly teach "speech". 

However, Wilson teaches this limitation at col. 7, lines 19-35 (i.e. voice). 

It would have been obvious to one of ordinary skill of the art having the teaching of 
Yamaguchi, Toyama and Wilson at the time the invention was made to modify the system of 
Yamaguchi and Toyama to include the above limitation as taught by Wilson. 

One of ordinary skill in the art would be motivated to make this combination in order to 
amplify the efficiency of the searching engine in view of Wilson, as doing so would give the 
added benefit of providing the capabilities of the system to efficiently index, store, insert, delete, 
and search hundreds of thousands or millions or records as taught by Wilson (col. 7, lines 19-35). 



Application/Control Number: 09/995,292 
Art Unit: 2167 



Page 9 



Response to Arguments 

5. Applicant's arguments filed 03/10/2003 have been fully considered but they are not 
persuasive. 

A. Claims 1, 2, 4-8, 10-14, 16-23, 25 
1; All claims limitations shown: 

Yamaguchi teaches "including time information with the captured media, biometric, and 
database information associated with an individual to create a multi-modal chronological dossier 
of the individual, wherein the time information includes when the media, biometric, and database 
information is captured" (i.e. for each registered person, pieces of information including the 
captured image or extracted feature as the specific information of the registered person, the 
dictionary, the name of the registered person, the registration number, the registration date 
when these pieces of information are stored in the storage device 1 1, and the use situation such 
as the date/time and place of identification of the registered person, col 5, lines 31-54, col. 2, 
line 20 to col. 3, line 33; Figs. 2, 3A, 3B; and col. 5, line 61 to col. 6, line 3). 

In col. 2, lines 30-45, Yamaguchi reference states: 

"there is provided a personal identification apparatus comprising a storage device which 
stores specific information of each of registered persons, an extraction device which can extract 
specific information of an object person from the object person, a classification device which 
classifies whether the object person is included in the registered persons by comparing the 
pieces of specific information of the registered persons, which are stored in the storage device, 
with the specific information of the object person, which is extracted by the extraction device, 
and a registered information operation device which sorts an order of the pieces of specific 
information of the registered persons, which are stored in the storage device, in accordance 
with a use situation of the registered persons, or sets whether the registered persons are to be 
subjected to classification " (col. 2, lines 30-45). 

In col. 5, lines 30-42, Yamaguchi reference states: 

"The storage device 11 stores, for each registered person, pieces of information 
including the captured image or extracted feature as the specific information of the registered 
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person, the dictionary, the name of the registered person, the registration number, the 
registration date when these pieces of information are stored in the storage device 1 1, and the 
use situation such as the date/time and place of identification of the registered person . The 
pieces of information registered for each registered person have, e.g., the data format as shown 
in FIG. 2. At the initial time, the pieces of information are arranged in the order of, e.g., 
registration" (col.^5, lines 31-42). 

In col. 5, line 62 to col. 6, line 3, Yamaguchi reference states: 
More specificall y, the control determination device 13 determines the sorting method 
for the information stored in the registered information storage device 2, computes the time 
required for determination of sorting, or changes the items. The sorting device 14 receives the 
stored information to change the order of registered information, sorts the pieces of received 
information in accordance with the reference designated by the control determination device 13, 
and transmits the sorted information to the registered information storage device 2 (col. 5, line 
62 to col. 6, line 3). 

Accordingly, "database information" equates to specific information of each of 
registered persons of Yamaguchi (See col. 2, lines 30-45). 

"an individual" equates to an object person of Yamaguchi (See col. 2, lines 30-45). 

Note that that Yamaguchi teaches the control determination device 13 determines the 
sorting method for the information stored in the registered information storage device 2, and the 
sorting device 14 receives the stored information to change the order of registered information, 
(See col. 2, lines 30-45). Furthermore, the date/time is included in the pieces of information of 
identification of the registered person (See col. 5, lines 31-42), and the pieces of information are 
arranged in the order (See col. 5, lines 31-42). 

Therefore, contrary to Applicant's argument, the Examiner asserts that Yamaguchi 

teaches the limitation "multi-modal chronological dossier of an individual" as Yamaguchi 

teaches a dossier or a history of a person in these recited paragraphs. 



2. No Motivation to Combine Yamaguchi with Toyama. 

In response to applicant's argument that there is no suggestion to combine the references, 
the examiner recognizes that obviousness can only be established by combining or modifying the 
teachings of the prior art to produce the claimed invention where there is some teaching, 
suggestion, or motivation to do so found either in the references themselves or in the knowledge 
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generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 
USPQ2d 1596 (Fed. Cir. 1988)and In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). 

In this case, Yamaguchi discloses a personal identification method comprising storing 
specific information (or images) of each of registered persons in a storage device, extracting 
specific information of an object person from the object person, classifying whether the object 
person is included in the registered persons by verifying the pieces of specific information of the 
registered persons, which are stored in the storage device, with the specific information of the 
object person, which is extracted in the extracting step, and performing operation of sorting an 
order of the pieces of specific information of the registered persons, which are stored in the 
storage device, in accordance with a use situation of the registered persons, or setting whether 
the registered persons are to be subjected to classification (Summary). 

Toyama discloses a system and method of providing an image-sharing environment that 
enhances manual and automatic image annotation (Summay). 

As discussed above, Yamaguchi teaches "an object person" for storing biometric 
information of a person, fingerprint, voiceprint, hair, or nail that can also be used as specific 
information (See col. 3, line 63 to col. 4, line 4), and captured image (See col. 5, lines 31-42). 
The information in the storage as disclosed in a table of Fig. 2, including an extra column (i.e. 
others, See Fig. 2), therefore, it would have been obvious to one of ordinary skill of the art to 
modify this extra column to store some annotated information. 

Both Yamaguchi and Toyama direct to a database for storing images, Toyama teaches the 
system including manual and automatic photo annotation, it would have been obvious to one of 
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ordinary skill of the art having the teaching of Yamaguchi and Toyama at the time the invention 
was made to modify the system of Yamaguchi to include this limitation as taught by Toyama. 

One of ordinary skill in the art would be motivated to make this combination in order to 
detect, compare multiple images of the same object and combine metadata from the various 
sources to better annotate each of the images as taught by Toyama (col. 5, lines 1-13), as doing 
so would give the added benefit of readily allowing a user to effectively search and control 
access to the image index or image database in view of Toyama (col. 4, lines 53-67). 

Thus, it is evident that claims 1, 2, 4-8, 10-14, 16-23, 25 are obvious in view of 
Yamaguchi and Toyama and it would have been obvious to combine Yamaguchi into Toyama in 
accordance with the motivation set forth above. 

3. Summary 

In response to the Applicant's argument regarding claim rejection under 35 U.S.C 103(a), 
as detailed, the claim language as presented is still read on by the Yamaguchi reference at the 
cited paragraph in the claim rejections. Yamaguchi does teach all Applicant's claim limitation 
including "multi-modal chronological dossier" of an individual. Further, the teaching of 
annotating'the captured information being taught by Toyama is used in combining with the 
system of Yamaguchi to render obvious the claimed limitation, despite the fact that applicant has 
recognized another advantage which would flow naturally from following the suggestion of the 
prior art that cannot be the basis for patentability when the differences would otherwise be 
obvious. See Ex parte Obiaya, 227 USPQ 58, 60 (Bd. Pat. App. & Inter. 1985). 
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B. Claim 9 

As pointed out by the examiner above, a case for obviousness is established by 
combining the Yamaguchi reference with Toyama as the combination teaches all the claim 
limitations. 

Although Yamaguchi and Toyama do not explicitly teach "speech", Wilson, however, 
discloses a method and apparatus for expandable biometric searching that can advantageously 
initiate searching in single or multi-dimensional space for multi-dimensional biometric data 
searching capabilities, including different types of biometric data such as fingerprint, facial 
shape, hand geometry, iris, retina, and/or voice (Summary). 

As the three references direct to methods and systems for retrieving, detecting, comparing 
images, it would have been obvious to one having ordinary skill in the art at the time of the 
invention to combine Yamaguchi, Toyama, and Wilson in arriving at the instant invention because, 
as Yamaguchi discloses in Summary, his invention may be used to store specific information of 
each of registered persons, extract specific information of an object person from the object 
person, and classify whether the object person is included in the registered persons by comparing 
the pieces of specific information of the registered persons. 

In addition, Toyama further provides the ability to assist the user in annotating the captured 
information. 

By combining Yamaguchi and Toyama, the user could readily and easily detect, compare 
multiple images of the same object and combines metadata from the various sources to better 
annotate each of the images. This would enhance the ability to effectively search and control 
access to the image index or image database. 
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Furthermore, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to add Wilson's multi-dimensional biometric data searching capabilities including voice 
because it would provide an improved method of efficiently searching for biometric data in a 
relatively large database of biometric data that having increased performance capabilities, e.g., 
speed of data searching and processing, and increased accuracy as taught by Wilson (col. 7, lines 
19-35). 

For all of the above reasons, arguments as raised are moot since all claim limitations 
relevant to this issue have been addressed accordingly. 

Conclusion 

6. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Miranda Le whose telephone number is (571) 272-41 12. The 
examiner can normally be reached on Monday through Friday from 8:30 AM to 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham, can be reached on (571) 272-7079. The fax number to this Art 
Unit is 571-273-8300. 

Any inquiry of a general nature or relating to the status of this application should be 
directed to the Group receptionist whose telephone number is (703) 305-3900. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http :// pair-direct . uspto . gov ■ Should you have questions on access to the Private 
PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

Miranda Le 
March 16, 2007 




